11 research outputs found
3D Shape Segmentation with Projective Convolutional Networks
This paper introduces a deep architecture for segmenting 3D objects into
their labeled semantic parts. Our architecture combines image-based Fully
Convolutional Networks (FCNs) and surface-based Conditional Random Fields
(CRFs) to yield coherent segmentations of 3D shapes. The image-based FCNs are
used for efficient view-based reasoning about 3D object parts. Through a
special projection layer, FCN outputs are effectively aggregated across
multiple views and scales, then are projected onto the 3D object surfaces.
Finally, a surface-based CRF combines the projected outputs with geometric
consistency cues to yield coherent segmentations. The whole architecture
(multi-view FCNs and CRF) is trained end-to-end. Our approach significantly
outperforms the existing state-of-the-art methods in the currently largest
segmentation benchmark (ShapeNet). Finally, we demonstrate promising
segmentation results on noisy 3D shapes acquired from consumer-grade depth
cameras.Comment: This is an updated version of our CVPR 2017 paper. We incorporated
new experiments that demonstrate ShapePFCN performance under the case of
consistent *upright* orientation and an additional input channel in our
rendered images for encoding height from the ground plane (upright axis
coordinate values). Performance is improved in this settin
Efficient Deduplication and Leakage Detection in Large Scale Image Datasets with a focus on the CrowdAI Mapping Challenge Dataset
Recent advancements in deep learning and computer vision have led to
widespread use of deep neural networks to extract building footprints from
remote-sensing imagery. The success of such methods relies on the availability
of large databases of high-resolution remote sensing images with high-quality
annotations. The CrowdAI Mapping Challenge Dataset is one of these datasets
that has been used extensively in recent years to train deep neural networks.
This dataset consists of 280k training images and 60k testing
images, with polygonal building annotations for all images. However, issues
such as low-quality and incorrect annotations, extensive duplication of image
samples, and data leakage significantly reduce the utility of deep neural
networks trained on the dataset. Therefore, it is an imperative pre-condition
to adopt a data validation pipeline that evaluates the quality of the dataset
prior to its use. To this end, we propose a drop-in pipeline that employs
perceptual hashing techniques for efficient de-duplication of the dataset and
identification of instances of data leakage between training and testing
splits. In our experiments, we demonstrate that nearly 250k(90%)
images in the training split were identical. Moreover, our analysis on the
validation split demonstrates that roughly 56k of the 60k images also appear in
the training split, resulting in a data leakage of 93%. The source code used
for the analysis and de-duplication of the CrowdAI Mapping Challenge dataset is
publicly available at https://github.com/yeshwanth95/CrowdAI_Hash_and_search .Comment: 9 pages, 2 figure
FacadeNet: Conditional Facade Synthesis via Selective Editing
We introduce FacadeNet, a deep learning approach for synthesizing building
facade images from diverse viewpoints. Our method employs a conditional GAN,
taking a single view of a facade along with the desired viewpoint information
and generates an image of the facade from the distinct viewpoint. To precisely
modify view-dependent elements like windows and doors while preserving the
structure of view-independent components such as walls, we introduce a
selective editing module. This module leverages image embeddings extracted from
a pre-trained vision transformer. Our experiments demonstrated state-of-the-art
performance on building facade generation, surpassing alternative methods
Learning Material-Aware Local Descriptors for 3D Shapes
Material understanding is critical for design, geometric modeling, and
analysis of functional objects. We enable material-aware 3D shape analysis by
employing a projective convolutional neural network architecture to learn
material- aware descriptors from view-based representations of 3D points for
point-wise material classification or material- aware retrieval. Unfortunately,
only a small fraction of shapes in 3D repositories are labeled with physical
mate- rials, posing a challenge for learning methods. To address this
challenge, we crowdsource a dataset of 3080 3D shapes with part-wise material
labels. We focus on furniture models which exhibit interesting structure and
material variabil- ity. In addition, we also contribute a high-quality expert-
labeled benchmark of 115 shapes from Herman-Miller and IKEA for evaluation. We
further apply a mesh-aware con- ditional random field, which incorporates
rotational and reflective symmetries, to smooth our local material predic-
tions across neighboring surface patches. We demonstrate the effectiveness of
our learned descriptors for automatic texturing, material-aware retrieval, and
physical simulation. The dataset and code will be publicly available.Comment: 3DV 201
Recommended from our members
Cross-Shape Attention for Part Segmentation of 3D Point Clouds
We present a deep learning method that propagates point-wise feature representations across shapes within a collection for the purpose of 3D shape segmentation. We propose a cross-shape attention mechanism to enable interactions between a shape\u27s point-wise features and those of other shapes. The mechanism assesses both the degree of interaction between points and also mediates feature propagation across shapes, improving the accuracy and consistency of the resulting point-wise feature representations for shape segmentation. Our method also proposes a shape retrieval measure to select suitable shapes for cross-shape attention operations for each test shape. Our experiments demonstrate that our approach yields state-of-the-art results in the popular PartNet dataset
Recurring Part Arrangements in Shape Collections
Extracting semantically related parts across models remains challenging, especially without supervision. The common approach is to co-analyze a model collection, while assuming the existence of descriptive geometric features that can directly identify related parts. In the presence of large shape variations, common geometric features, however, are no longer sufficiently descriptive. In this paper, we explore an indirect top-down approach, where instead of part geometry, part arrangements extracted from each model are compared. The key observation is that while a direct comparison of part geometry can be ambiguous, part arrangements, being higher level structures, remain consistent, and hence can be used to discover latent commonalities among semantically related shapes. We show that our indirect analysis leads to the detection of recurring arrangements of parts, which are otherwise difficult to discover in a direct unsupervised setting. We evaluate our algorithm on ground truth datasets and report advantages over geometric similarity-based bottom-up co-segmentation algorithms. 1
Data Leakage Detection and De-duplication in Large Scale Image Datasets
Auxilliary files for the paper "Data Leakage Detection and De-duplication in Large Scale Image Datasets"Please read README.md for a detailed description and instructions on how to use these auxilliary files.</p